Is it feasible and effective to provide faculty development programs online for clinical teachers? / 한국의학교육

Purpose@#Seoul National University College of Medicine operates a faculty development program for clinical teachers at multiple affiliated teaching hospitals. In 2020, the program was moved online due to coronavirus disease 2019. The purpose of this study was to determine whether it is feasible and effective to provide faculty development programs online in terms of clinical teachers’ participation and satisfaction in comparison with offline programs. @*Methods@#Clinical teachers participated in the clinical teaching methods programs offline in 2019 and online in 2020. We analyzed participation rate and satisfaction level. All surveys items were rated on a 5-point Likert scale. We also interviewed instructors about the advantages and drawbacks of the online program. @*Results@#The participation rate of the online program (89.5%) was significantly higher than that of the offline program (67.8%). The overall satisfaction level for the online program (4.37) was similar to that for the offline program (4.50). @*Conclusion@#Faculty development programs online are feasible and effective in medical education. We need to design training content that fits online programs, consider various online training methods to reinforce the strengths of online programs, and support participants to make good use of these programs.

Is it feasible and effective to provide faculty development programs online for clinical teachers? / 한국의학교육

Purpose@#Seoul National University College of Medicine operates a faculty development program for clinical teachers at multiple affiliated teaching hospitals. In 2020, the program was moved online due to coronavirus disease 2019. The purpose of this study was to determine whether it is feasible and effective to provide faculty development programs online in terms of clinical teachers’ participation and satisfaction in comparison with offline programs. @*Methods@#Clinical teachers participated in the clinical teaching methods programs offline in 2019 and online in 2020. We analyzed participation rate and satisfaction level. All surveys items were rated on a 5-point Likert scale. We also interviewed instructors about the advantages and drawbacks of the online program. @*Results@#The participation rate of the online program (89.5%) was significantly higher than that of the offline program (67.8%). The overall satisfaction level for the online program (4.37) was similar to that for the offline program (4.50). @*Conclusion@#Faculty development programs online are feasible and effective in medical education. We need to design training content that fits online programs, consider various online training methods to reinforce the strengths of online programs, and support participants to make good use of these programs.

The Effect of the Dr. LEE Jong-wook—Seoul Project on Medical Student Academic Performance in Laos / 의학교육논단

The Dr. LEE Jong-wook—Seoul Project (DLSP) in the Lao People'0s Democratic Republic (PDR) is a long-term development cooperation initiative aimed at upgrading the medical education capacity at the University of Health Sciences (UHS) in the Lao PDR. Under the project, eight faculty members from UHS were trained at the Seoul National University College of Medicine for a 12-month period during 2010–2011. Using a training-of-trainers model, we estimated the effect of DLSP on the learning outcomes of students, which was measured using the standard test scores of the students. The questionnaire developed by the Medical Education Assessment Consortium in Korea was given to the students and both the percent-correct and standardized scores were examined. The evaluation strategy was to compare the learning outcomes of the students taught by the trainees with those that were not and to compare the medical topics taught by the trainees with other topics. It was found that the first group of trainees in the DLSP was associated with an increase of 1.5 in the percent-correct test scores for basic medicine. While the interpretation requires caution, it is consistent with the view that a training program for faculty members could be effective in improving the teaching capacity of medical schools in developing countries.

Applying a mixed methods design for needs assessment of an international fellowship training program for Mongolian health professions / 한국의학교육

PURPOSE: Continuing professional development is essential for improving health care services, especially in developing countries. Most of the training programs in Mongolia were not based on a targeted needs assessment. Therefore, we aimed to apply a mixed methods design to assess the training needs of Mongolian health professions. METHODS: We conducted a needs assessment using a convergent parallel mixed methods design in two steps. The survey and interview questions were developed to identify priority areas, targeted trainees, and effective training methods. A survey on 60 respondents, 15 individual interviews, and a focus group interview with 14 participants were conducted in the first step. In the second step, 12 representatives of key stakeholders were invited to a second focus group interview. RESULTS: Current health policy areas, areas related to future national plans, and areas not currently receiving governmental or international support were suggested as the main priorities. The stakeholder suggested that trainees should be selected based on their professional experience and language level, as well as each hospital’s needs. Building teams including various professions, such as nurses, technicians, and biomedical engineers, was recommended as a way to exchange ideas with each other and to build teamwork for future collaboration. CONCLUSION: Medical training needs are dynamic and complex; therefore, a deep understanding of the context and setting is necessary. In this study, we assessed the targeted training needs of Mongolian health professions through a mixed methods design, which could be an effective way to conduct needs assessments for training programs.

Investigating the effects of interprofessional communication education for medical students / 한국의학교육

PURPOSE: Interprofessional communication skills are an essential competency for medical students training to be physicians. Nevertheless, interprofessional education (IPE) is relatively rare in Korean medical schools compared with those overseas. We attempted to evaluate the effectiveness of the first IPE program in our school. METHODS: In the first semester of the school year 2018, third-grade medical students (N=149) at the Seoul National University College of Medicine participated in ‘communication between healthcare professionals in the clinical field’ training, which consisted of small group discussions and role-play. To evaluate the effectiveness of this training, we conducted pre- and post-training questionnaire surveys. Comparing paired t-tests, we evaluated the students' competency in interpersonal communication and their attitude towards the importance of IPE before and after the training. The Global Interpersonal Communication Competence Scale (GICC-15) was used to evaluate competency in interpersonal communication. RESULTS: Out of 149 students, 144 completed the pre- and post-training questionnaires. The total GICC-15 scores before and after training were 55.60±6.94 (mean±standard deviation) and 58.89±7.34, respectively (p=0.000). All subcategory scores of GICC-15 after training were higher after training and were statistically significant (p<0.05), except for two subcategories. The importance of IPE score also improved after training but was not significant (p=0.159). The appropriateness of content and training method scores were 3.99±0.92 and 3.94±1.00, respectively. CONCLUSION: From the results, our school's IPE program demonstrated a positive overall educational effect. Deployment of systematic and varied IPE courses is expected in the future, with more longitudinal evaluation of educational effect.

Applying a mixed methods design for needs assessment of an international fellowship training program for Mongolian health professions / 한국의학교육

PURPOSE@#Continuing professional development is essential for improving health care services, especially in developing countries. Most of the training programs in Mongolia were not based on a targeted needs assessment. Therefore, we aimed to apply a mixed methods design to assess the training needs of Mongolian health professions.@*METHODS@#We conducted a needs assessment using a convergent parallel mixed methods design in two steps. The survey and interview questions were developed to identify priority areas, targeted trainees, and effective training methods. A survey on 60 respondents, 15 individual interviews, and a focus group interview with 14 participants were conducted in the first step. In the second step, 12 representatives of key stakeholders were invited to a second focus group interview.@*RESULTS@#Current health policy areas, areas related to future national plans, and areas not currently receiving governmental or international support were suggested as the main priorities. The stakeholder suggested that trainees should be selected based on their professional experience and language level, as well as each hospital’s needs. Building teams including various professions, such as nurses, technicians, and biomedical engineers, was recommended as a way to exchange ideas with each other and to build teamwork for future collaboration.@*CONCLUSION@#Medical training needs are dynamic and complex; therefore, a deep understanding of the context and setting is necessary. In this study, we assessed the targeted training needs of Mongolian health professions through a mixed methods design, which could be an effective way to conduct needs assessments for training programs.

Investigating the effects of interprofessional communication education for medical students / 한국의학교육

PURPOSE@#Interprofessional communication skills are an essential competency for medical students training to be physicians. Nevertheless, interprofessional education (IPE) is relatively rare in Korean medical schools compared with those overseas. We attempted to evaluate the effectiveness of the first IPE program in our school.@*METHODS@#In the first semester of the school year 2018, third-grade medical students (N=149) at the Seoul National University College of Medicine participated in ‘communication between healthcare professionals in the clinical field’ training, which consisted of small group discussions and role-play. To evaluate the effectiveness of this training, we conducted pre- and post-training questionnaire surveys. Comparing paired t-tests, we evaluated the students' competency in interpersonal communication and their attitude towards the importance of IPE before and after the training. The Global Interpersonal Communication Competence Scale (GICC-15) was used to evaluate competency in interpersonal communication.@*RESULTS@#Out of 149 students, 144 completed the pre- and post-training questionnaires. The total GICC-15 scores before and after training were 55.60±6.94 (mean±standard deviation) and 58.89±7.34, respectively (p=0.000). All subcategory scores of GICC-15 after training were higher after training and were statistically significant (p<0.05), except for two subcategories. The importance of IPE score also improved after training but was not significant (p=0.159). The appropriateness of content and training method scores were 3.99±0.92 and 3.94±1.00, respectively.@*CONCLUSION@#From the results, our school's IPE program demonstrated a positive overall educational effect. Deployment of systematic and varied IPE courses is expected in the future, with more longitudinal evaluation of educational effect.

How to execute Context, Input, Process, and Product evaluation model in medical health education / 보건의료교육평가

Improvements to education are necessary in order to keep up with the education requirements of today. The Context, Input, Process, and Product (CIPP) evaluation model was created for the decision-making towards education improvement, so this model is appropriate in this regard. However, application of this model in the actual context of medical health education is considered difficult in the education environment. Thus, in this study, literature survey of previous studies was investigated to examine the execution procedure of how the CIPP model can be actually applied. For the execution procedure utilizing the CIPP model, the criteria and indicators were determined from analysis results and material was collected after setting the material collection method. Afterwards, the collected material was analyzed for each CIPP element, and finally, the relationship of each CIPP element was analyzed for the final improvement decision-making. In this study, these steps were followed and the methods employed in previous studies were organized. Particularly, the process of determining the criteria and indicators was important and required a significant effort. Literature survey was carried out to analyze the most widely used criteria through content analysis and obtained a total of 12 criteria. Additional emphasis is necessary in the importance of the criteria selection for the actual application of the CIPP model. Also, a diverse range of information can be obtained through qualitative as well as quantitative methods. Above all, since the CIPP evaluation model execution result becomes the basis for the execution of further improved evaluations, the first attempt of performing without hesitation is essential.

Evaluation of Medical Humanities Course in College of Medicine Using the Context, Input, Process, and Product Evaluation Model

BACKGROUND: Seoul National University College of Medicine has provided a new educational curriculum on basis of the competency-based curriculum since 2016. The new curriculum included the medical humanities course (MHC) to potentiate humanities of medical students. The present study applied the context, input, process and product (CIPP) evaluation model to the MHC in order to confirm the feasibility of the CIPP model and to improve the MHC by questionnaire survey and analysis of teaching materials. METHODS: This study analyzed the MHC provided to the freshmen in 2016 and to the freshmen and sophomores in 2017 by the CIPP model. Firstly, evaluation criteria and indicators were developed according to the CIPP classification. The materials collected from the course were analyzed by quantitative and qualitative analysis according to the evaluation criteria. In the quantitative analysis, an independent sample t-test was performed to verify the difference in the responses between the students (n = 522) and the professors (n = 22). In addition, content analysis was conducted for qualitative evaluation. RESULTS: There were significant differences in perceptions of MHC between students and professors about the results of almost all objective survey questions through the t-test, such as score 3.64 in students and 4.48 in professors in response to the item of ‘provision of appropriate feedback.’ As a result of the content analysis, 7 categories and 20 subcategories were derived. There were the most responses to various instructional methods (students, 20%; professors, 21.5%). CONCLUSION: The CIPP evaluation model was acceptable for the MHC analysis. The first task is to raise students' awareness of the MHC in order to improve the MHC quality.

Developing a core competency model for translational medicine curriculum / 한국의학교육

PURPOSE: This study aimed to develop a core competency model for translational medicine curriculum in the Korean graduate education context. METHODS: We invited specialists and key stakeholders to develop a consensus on a core competency model. The working group composed of 17 specialists made an initial draft of a core competency model based on the literature review. The initial draft was sent to the survey group by email to ask whether they agreed or disagreed with each core competency. The working group simplified, merged, or excluded the competencies that received less than 80% agreement among the 43 survey respondents. The working group also reorganized the order of the domains and competencies based on the survey results, and clustered the domains into four major areas. RESULTS: The final core competency model has four areas, 12 domains, and 34 core competencies. The major areas are theory-based problem assessment and formulation, study design and measurement, study implementation, and literature review and critique. CONCLUSION: This new core competency model will provide guidance for the competency based education of translational medicine in Korea.

Evaluation of Process and Satisfaction for Selective Courses in a Medical School / 의학교육논단

The necessity of embracing selective courses in medical curriculum is increasing due to the expansion of medical knowledge and changes in the health care environment. In contrast to the abundant evidence regarding elective or selective courses during the clinical phase, articles focusing on the preclinical period are relatively scarce. This study aims to explore the development, implementation, and evaluation of newly-adopted selective courses in the first-year medical curriculum in a medical school which recently underwent a major curricular revision. First of all, the Curriculum Committee established goals and operating principles of the courses, and then the committee encouraged all participating professors to attend a related faculty development workshop after finalizing the list of courses. A survey was conducted at the end of each course for evaluation. Of the 36 courses opened in 2016, the overall satisfaction of students was 4.98±1.06 (out of 6) and showed a strong correlation with students' previous expectations, reasoning- and participation-oriented teaching, and outcome of the courses including increased motivation. In the open-ended responses, students and professors described not only intended outcomes such as acquisition of medical knowledge and increased interest in new topics, but also unintended outcomes including positive impression for selective courses and even high satisfaction and rewarding experiences, especially from the teachers' perspective. Although long-term outcomes remain to be seen, the results of this study show the feasibility and impact of selective courses and will contribute to effective implementation in other medical schools.

Evaluation of a continuing professional development training program for physicians and physician assistants in hospitals in Laos based on the Kirkpatrick model / 보건의료교육평가

PURPOSE: Medical professionals from Korea and Laos have been working together to develop a continuing professional development training program covering the major clinical fields of primary care. This study aimed to evaluate the effectiveness of the program from 2013 to 2014 using the Kirkpatrick model. METHODS: A questionnaire was used to evaluate the reaction of the trainees, and the trainers assessed the level of trainees' performance at the beginning and the end of each clinical section. The transfer (behavioral change) of the trainees was evaluated through the review of medical records written by the trainees before and after the training program. RESULTS: The trainees were satisfied with the training program, for which the average score was 4.48 out of 5.0. The average score of the trainees' performance at the beginning was 2.39 out of 5.0, and rose to 3.88 at the end of each section. The average score of the medical records written before the training was 2.92 out of 5.0, and it rose to 3.34 after the training. The number of patient visits to the district hospitals increased. CONCLUSION: The continuing professional development training program, which was planned and implemented with the full engagement and responsibility of Lao health professionals, proved to be effective.

Transnational collaboration for faculty development in health professions education in Mongolia / 한국의학교육

PURPOSE: The Mongolian National University of Medical Sciences is the only national university in Mongolia and has produced more than 90% of health professionals in the country. Experts from Mongolia and Korea embarked on a collaborative effort to develop educational programs for faculty development based on the personal and professional needs of faculty members. This study aimed to evaluate the outcomes of those educational programs to determine whether this transnational collaboration was successful. METHODS: A needs assessment survey was conducted among 325 faculty members. Based on the results of this survey, the joint expert team developed educational programs on seven core topics: clinical teaching, curriculum development, e-learning, item writing, medical research, organizational culture, and resident selection. Surveys evaluating the satisfaction and the attitudes of the participants were conducted for each program. RESULTS: Throughout the 17-day program, 16 experts from Korea and 14 faculty members from Mongolia participated as instructors, and a total of 309 participants attended the program. The average satisfaction score was 7.15 out of 8.0, and the attitudes of the participants towards relevant competencies significantly improved after each educational program. CONCLUSION: The faculty development programs that were developed and implemented as part of this transnational collaboration between Mongolia and Korea are expected to contribute to the further improvement of health professions education in Mongolia. Future studies are needed to evaluate the long-term outcomes of these educational programs.

The Effect of Formative Program Evaluation on Continuous Program Improvement: A Case Study of a Clinical Training Program in Lao PDR

This study aimed to evaluate the effect of the formative program evaluation on the continuous improvement of a clinical training program for Lao health professionals. The training program was conducted 4 times consecutively for total 48 health professionals, and the formative program evaluation was carried out during the whole process. To evaluate the satisfaction and the transfer of the trainees, the questionnaire survey, the focus group interview, and the trainees' medical records were used. After the end of each batch of the program, the evaluation data were analyzed, and its results were shared with the training management committee and the trainers, who, based on the results, reached a consensus on how to improve the program. The evaluation results and the comparison of them among the four batches of the program showed that there was a continuous increase of the satisfaction and the transfer of the trainees, especially in the early period of the program. The formative program evaluation which was conducted during the whole process of the clinical training program had a positive effect on the improvement of the program, especially in the early phase, by increasing the satisfaction and transfer of the trainees.

What is the current orientation of undergraduate medical education in Korea? / 한국의학교육

PURPOSE: The educational purpose of a medical school is important, because it guides educational decisions in an individual organization and projects the image of the doctors that we are generating. By analyzing the educational goals of entire medical schools, this study aimed to examine the current orientation and future direction of undergraduate medical education in Korea. METHODS: Educational goals were collected from the website of each medical school and subjected to inductive content analysis. After identifying categories and themes, we examined the differences between medical school subgroups and compared the categories with competencies that have been suggested by the "Korean Doctor's Role." RESULTS: Thirteen themes were identified: medical expertise, professionalism, contribution to various levels of society, self-management and development, basic educational ideology, research ability, cooperation, leadership, dealing with future change, respect for life, creativity, problem-solving ability, and ability to educate. There was a significant difference in educational goals between medical schools when grouped by geographic location and affiliation of research-driven hospitals. Of the 16 competencies that are suggested by the Korean Doctor's Role, 12 had one or more corresponding categories. CONCLUSION: Per their current educational purposes, Korean medical schools pursue a broad variety of competencies that need cultivating during the course of undergraduate medical education. Further research is needed to determine how best to apply these educational purposes in actual institutions and ultimately lead them to become part of the competency of a graduate.

Research and development of evaluation criteria for premedical curriculum / 한국의학교육

PURPOSE: The purpose of this study was to develop criteria to evaluate a premedical curriculum to ultimately improve the quality of premedical education. METHODS: The first draft of the evaluation criteria was developed through a literature review and expert consultation. The Delphi survey was conducted to ensure the validity of the draft. RESULTS: The final premedical curriculum criteria consisted of three evaluation areas (curriculum development, curriculum implementation, and curriculum outcome), five evaluation items (educational objective, organization of curriculum, instructional method, class management, and educational outcome), and 18 evaluation indicators. CONCLUSION: There should be further discussion on the evaluation questionnaire and the content for each evaluation indicator with regard to its practical application. Also, a concrete evaluation system, including evaluation standards and rating scales, should be developed.

Evaluation of first year outcomes of Lee Jong-Wook Seoul Project using Kirkpatrick's four-level evaluation approach / 한국의학교육

PURPOSE: The Lee Jong-Wook Seoul Project, which began in 2011, is a training program that aims to educate faculty members of Lao University of Health Sciences in medical knowledge and skills. The project has tremendous significance, attempting to apply the Korean health care developmental model to developing countries. Precise evaluation of the project outcome must be performed to enhance the effectiveness, improve the shortcomings, and adjust the future direction of the program. METHODS: Trainers and exchange professors were asked to fill out questionnaires in the middle and at the end of the program. Eight months after its completion, an evaluation team visited Lao and interviewed each exchange professor individually. Also, 360-degree feedback was obtained from their colleagues. We analyzed the answers to the questionnaires, based on Kirkpatrick's 4-level model. RESULTS: The trainers and exchange professors had positive reactions to the program. All participants felt that their knowledge and skills improved (in their respective fields), although the actual improvement was not extensive, according to their trainers' assessments. The 360-degree feedback demonstrated that the participants actively passed on their experience, knowledge, and skills to their colleagues on returning to work. CONCLUSION: Although a 4-level evaluation was not conducted and despite the limitations in examining academic achievement, interviewing Lao professors, and acquiring quantitative data at Lao University of Health Sciences, this project has demonstrated its value in the development of individual professors and their colleagues.

How different are premedical freshmen who enter after introducing a multiple mini-interview in a medical school? / 한국의학교육

PURPOSE: Seoul National University College of Medicine (SNUMC) introduced a multiple mini-interview (MMI) to assess the noncognitive ability of applicants in 2013. This study aims to examine whether students differ with regard to their personal career values and academic achievements by admission type. METHODS: We administered a survey about career values and self-perception of competencies. We then compared the survey results and academic achievements by admission type. Finally, the correlation coefficient between MMI score and academic achievement was calculated in the MMI group. The data were analyzed by t-test and correlation analysis. RESULTS: There was no statistically significant difference in career values between groups. For self-perception of competency, only the 'interact in heterogeneous groups' domain was higher for those who entered through the MMI. The MMI group had a higher and broader level of academic achievement. Within the MMI group, there was a significant correlation between grade point average and MMI station scores. CONCLUSION: The characteristics of students who entered through the MMI were more in accordance with the goals of SNUMC and the competency of future doctors. Considering the unique feature of premedical academic achievement, this result implies that such students have superior noncognitive abilities, such as self-development and internal motivation. How these differences change subsequently remains to be seen.

Clinical-Performance Remediation Program for Dyscompetent Medical Students / 한국의학교육

PURPOSE: Medical schools endeavor to ensure that students are competent with regard to clinical skills. Skills remediation is implemented in cases of poor clinical performance examination (CPX) grades, although little is known about the effectiveness of such techniques. In this study, we examined the effectiveness of a remediation program that was designed to improve the clinical performance of medical students. METHODS: A 6-week remediation program, administered jointly by Seoul National University College of Medicine's Departments of Internal Medicine (IM) and Family Medicine (FM), was initiated. The program was divided into 2 parts: 3 weeks each of IM classes that were run by specialists in various fields and FM classes that were conducted by a chief resident. Twenty-three students were required to undergo remediation after posting poor scores on 2 sessions of a CPX. On completion of the remediation program, the students' clinical performance was re-evaluated, and the changes in clinical performance scores were analyzed. RESULTS: After the remediation program, the students' total scores and scores on history taking, physical examination, physician's manner, and physician-patient interaction improved significantly. However, patient education did not improve. Most students found the remediation program to be instructive and helpful in preparation for the CPX. They were more satisfied with the chief resident's serial tutoring than with specialists' tutoring sessions. CONCLUSION: The remediation program improves clinical performance. Continued development and implementation of this program will help failing students be competent physicians.

A study on premedical curriculum reform of one medical school / 한국의학교육

PURPOSE: The aim of this study was to examine the background and improvement of the reformed premedical curriculum in Seoul National University and to analyze in which it corresponds with its intentions. It gives implications to premedical curriculum through its development model. METHODS: The background and improvement of the reformed premedical curriculum was analyzed through several reports from those of associated committees. The development model of the reformed curriculum was made in order to examine correspondence with the intentions of its reformation. RESULTS: The graduate credit increased while compulsory credit was in decrease which leads to the elimination of standardized education based on the natural sciences. The requirements in compulsory liberal arts and elective major subjects were suggested in order to develop students' perspectives in humanities. CONCLUSION: As premed is a preliminary course before studying medicine, premedical curriculum should reflect the needs of those parties concerned-society, professors, students, etc.- and be based on core values and educational philosophy of the school in order to derive its competences.